{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "DRFDEL14XTiN"
   },
   "source": [
    "# Agent Observability with OpenLIT\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "PqhddfHZXgVH"
   },
   "source": [
    "\n",
    "\n",
    "<img src=\"https://github.com/openlit/.github/blob/main/profile/assets/wide-logo-no-bg.png?raw=true\" alt=\"OpenLIT Logo for LLM Observability\" width=\"30%\"></img>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "dmNFkiaGYXIv"
   },
   "source": [
    "[OpenLIT](https://github.com/openlit/openlit) an open source product that helps developers build and manage AI agents in production, effectively helping them improve accuracy. As a self-hosted solution, it enables developers to experiment with LLMs, manage and version prompts, securely manage API keys, and provide safeguards against prompt injection and jailbreak attempts. It also includes built-in OpenTelemetry-native observability and evaluation for the complete GenAI stack (LLMs, Agents, vector databases, and GPUs).\n",
    "\n",
    "For more info, check out the [OpenLIT Repo](https://github.com/openlit/openlit)\n",
    "\n",
    "![](https://github.com/openlit/.github/blob/main/profile/assets/openlit-client-1.png?raw=true)\n",
    "![](https://github.com/openlit/.github/blob/main/profile/assets/openlit-client-2.png?raw=true)\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "hePjFDc6Yu6G"
   },
   "source": [
    "## Adding OpenLIT to an existing AG2 (Now AG2) service\n",
    "To get started, you'll need to install the OpenLIT library\n",
    "\n",
    "OpenLIT uses OpenTelemetry to automatically instrument the AI Agent app when it's initialized meaning your agent observability data like execution traces and metrics will be tracked in just one line of code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kyb8oFmCZdg8",
    "outputId": "3e34e1ef-039b-4d38-f66b-67f145cc5a24"
   },
   "outputs": [],
   "source": [
    "! pip install -U ag2[openai] openlit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pxawniRGYyj2"
   },
   "outputs": [],
   "source": [
    "import openlit\n",
    "\n",
    "from autogen import AssistantAgent, LLMConfig, UserProxyAgent\n",
    "\n",
    "openlit.init()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "hK4KL2JLZwLg"
   },
   "source": [
    "OpenLIT will now start automatically tracking\n",
    "\n",
    "- LLM prompts and completions\n",
    "- Token usage and costs\n",
    "- Agent names and actions\n",
    "- Tool usage\n",
    "- Errors\n",
    "\n",
    "\n",
    "## Lets look at a simple chat example\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "Z_5QJzEqa7I0"
   },
   "outputs": [],
   "source": [
    "import openlit\n",
    "\n",
    "openlit.init()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "2CKZPle2aal6",
    "outputId": "7f3346e9-7e5b-4c47-f111-32f8d06313a0"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "llm_config = LLMConfig(config_list={\"model\": \"gpt-4\", \"api_key\": os.environ[\"OPENAI_API_KEY\"]})\n",
    "assistant = AssistantAgent(\"assistant\", llm_config=llm_config)\n",
    "user_proxy = UserProxyAgent(\"user_proxy\", code_execution_config=False)\n",
    "\n",
    "# Start the chat\n",
    "user_proxy.initiate_chat(\n",
    "    assistant,\n",
    "    message=\"Tell me a joke about NVDA and TESLA stock prices.\",\n",
    ")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {
    "id": "7_gNWze0bYdO"
   },
   "source": [
    "# Sending Traces and metrics to OpenLIT\n",
    "\n",
    "By default, OpenLIT generates OpenTelemetry traces and metrics that are logged to your console. To set up a detailed monitoring environment, this guide outlines how to deploy OpenLIT and direct all traces and metrics there. You also have the flexibility to send the telemetry data to any OpenTelemetry-compatible endpoint, such as Grafana, Jaeger, or DataDog.\n",
    "\n",
    "## Deploy OpenLIT Stack\n",
    "\n",
    "1. Clone the OpenLIT Repository\n",
    "\n",
    "   Open your terminal or command line and execute:\n",
    "\n",
    "   ```shell\n",
    "   git clone git@github.com:openlit/openlit.git\n",
    "   ```\n",
    "\n",
    "2. Host it Yourself with Docker\n",
    "\n",
    "   Deploy and start OpenLIT using the command:\n",
    "\n",
    "   ```shell\n",
    "   docker compose up -d\n",
    "   ```\n",
    "\n",
    "> For instructions on installing in Kubernetes using Helm, refer to the [Kubernetes Helm installation guide](https://docs.openlit.io/latest/installation#kubernetes).\n",
    "\n",
    "Configure the telemetry data destination as follows:\n",
    "\n",
    "| Purpose                                   | Parameter/Environment Variable                   | For Sending to OpenLIT         |\n",
    "|-------------------------------------------|--------------------------------------------------|--------------------------------|\n",
    "| Send data to an HTTP OTLP endpoint        | `otlp_endpoint` or `OTEL_EXPORTER_OTLP_ENDPOINT` | `\"http://127.0.0.1:4318\"`      |\n",
    "| Authenticate telemetry backends           | `otlp_headers` or `OTEL_EXPORTER_OTLP_HEADERS`   | Not required by default        |\n",
    "\n",
    "> 💡 Info: If the `otlp_endpoint` or `OTEL_EXPORTER_OTLP_ENDPOINT` is not provided, the OpenLIT SDK will output traces directly to your console, which is recommended during the development phase.\n",
    "\n",
    "## Visualize and Optimize!\n",
    "\n",
    "With the Observability data now being collected and sent to OpenLIT, the next step is to visualize and analyze this data to get insights into your AI application's performance, behavior, and identify areas of improvement.\n",
    "\n",
    "Just head over to OpenLIT at `127.0.0.1:3000` on your browser to start exploring. You can login using the default credentials\n",
    "  - **Email**: `user@openlit.io`\n",
    "  - **Password**: `openlituser`\n",
    "\n",
    "![](https://github.com/openlit/.github/blob/main/profile/assets/openlit-ag2-1.png?raw=true)\n",
    "![](https://github.com/openlit/.github/blob/main/profile/assets/openlit-ag2-2.png?raw=true)"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "front_matter": {
   "description": "Use OpenLIT to easily monitor AI agents in production with OpenTelemetry.",
   "tags": [
    "integration",
    "monitoring",
    "observability",
    "debugging"
   ]
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
